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(57) ABSTRACT 

A method of providing weather radar images to a user 
includes obtaining radar image data corresponding to a 
weather radar image to be displayed. The radar image data 
is image processed to identify a feature of the weather radar 
image which is potentially indicative of a hazardous weather 
condition. The weather radar image is displayed to the user 
along with a notification of the existence of the feature 
which is potentially indicative of the hazardous weather 
condition. Notification can take the form of textual infor- 
mation regarding the feature, including feature type and 
proximity information. Notification can also take the form of 
visually highlighting the feature, for example by forming a 
visual border around the feature. Other forms of notification 
can also be used. 

18 Claims, 7 Drawing Sheets 



120 


110 



CONTROLLER 


WARNING 

DEVICE 


145 


130 









100 


U.S. Patent Nov. 18,2003 


Sheet 1 of 7 


US 6,650,275 B1 



o 






U.S. Patent Nov. 18,2003 


Sheet 2 of 7 


US 6,650,275 B1 





U.S. Patent Nov. 18,2003 


Sheet 3 of 7 


US 6,650,275 B1 















U.S. Patent Nov. 18,2003 


Sheet 6 of 7 


US 6,650,275 B1 






U.S. Patent Nov. 18,2003 


Sheet 7 of 7 


US 6,650,275 B1 





US 6,650,275 B1 


1 

IMAGE PROCESSING FOR HAZARD 
RECOGNITION IN ON-BOARD WEATHER 
RADAR 

The U.S. Government has a paid-up license in this 5 
invention and the right in limited circumstances to require 
the patent owner to license others on reasonable terms as 
provided for by the terms of the contract number NCC1-290 
awarded by the NASA. 

FIELD OF THE INVENTION 10 

The present invention relates to on-board weather radar 
for use on aircraft or other vehicles. More particularly, the 
present invention relates to methods and apparatus for 
rendering hazardous condition features more recognizable 
by pilots or the operators of other vehicles. 

BACKGROUND OF THE INVENTION 

Weather has been identified as a cause or contributing 
factor to nearly 25% of aviation accidents and 35% of 
fatalities. Accidents occur when a chain of events leads to a 20 
failure of an aircraft system, a mistake on part of the crew 
piloting the aircraft, or a combination thereof. Improved 
levels of weather information and the use of pilot decision 
aids may be helpful in breaking the chain of events that leads 
to an accident. 25 

On-board weather radar is a tactical navigation aid that 
shows convective hazards for the next 15 to 30 minutes of 
flight. The on-board weather radar currently serves as the 
focus of a pilot’s awareness of the weather situation. It 
provides a graphical depiction of precipitation relative to the 30 
aircraft’s position. The presence of turbulence can be 
inferred from these reflections off of precipitation. Through 
the use of Doppler processing, turbulence can also be 
detected and displayed directly. 

While the on-board weather radar has proven invaluable 
as a real-time source of tactical weather information, it has 
limitations. For example, there are features of weather radar 
images that indicate hazardous conditions may exist. Since 
some of these features occur infrequently, pilots may not be 
able to maintain proficiency in correctly interpreting these 
features. Since some of these features are short-lived (i.e., 
are only displayed for a short period of time), pilots may not 
notice the potential hazard and may not take appropriate 
action. A method of increasing the likelihood that a pilot will ^ 
notice a potential hazard on the weather radar image, and 
thus take appropriate action, would be a significant improve- 
ment in the art. 

SUMMARY OF THE INVENTION 

A method of providing weather radar images to a user 50 
includes obtaining radar image data corresponding to a 
weather radar image to be displayed. The radar image data 
is image processed to identify a feature of the weather radar 
image which is potentially indicative of a hazardous weather 
condition. The weather radar image is displayed to the user 55 
along with a notification of the existence of the feature 
which is potentially indicative of the hazardous weather 
condition. Notification can take the form of textual infor- 
mation regarding the feature, including feature type and 
proximity information. Notification can also take the form of 60 
visually highlighting the feature, for example by forming a 
visual border around the feature. Other forms of notification 
can also be used. 

BRIEF DESCRIPTION OF THE DRAWINGS 65 

FIG. 1 is a block diagram illustrating an on-board weather 
radar system in accordance with the invention. 
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FIG. 2 is an illustration of the radar indicator of FIG. 1, 
illustrating a “squall line” hazardous weather condition 
feature and warning. 

FIG. 3 is an illustration of the radar indicator of FIG. 1, 
illustrating a “hook” hazardous weather condition feature 
and warning. 

FIG. 4 is an illustration of the radar indicator of FIG. 1, 
illustrating a “steep gradient” hazardous weather condition 
feature and warning. 

FIG. 5 is an illustration of the radar indicator of FIG. 1, 
illustrating a “scalloped edge” hazardous weather condition 
feature and warning. 

FIG. 6 is an illustration of the radar indicator of FIG. 1, 
illustrating a “finger” hazardous weather condition feature 
and warning. 

FIG. 7 is an illustration of the radar indicator of FIG. 1, 
illustrating a “horse-shoe shape” hazardous weather condi- 
tion feature and warning. 

DETAILED DESCRIPTION OF AN 
ILLUSTRATIVE EMBODIMENT 

FIG. 1 is a block diagram illustrating a weather radar 
system 100 in accordance with the invention. System 100 
includes radome 110, antenna (shown with mount) 115, 
receiver/transmitter 120, controller 130, hazardous feature 
image processor 135, radar indicator or display 140, and 
(optionally) warning device 145. Under the control of con- 
troller 130, receiver/transmitter 120 transmits a radar pulse 
through antenna 115 and radome 110 in one particular 
direction at a time. The radar pulse travels outward from the 
aircraft in the particular direction and reflects off of precipi- 
tation or other weather phenomena. As reflections come 
back and are received by antenna 115 and receiver/ 
transmitter 120, an indication of the weather at different 
distances (out to a range) along a straight line in the 
particular direction can be ascertained. 

The data that is received by receiver/transmitter 120 is put 
into a data stream and provided to image processor 135. As 
antenna 115 sweeps back and forth, data for hundreds of 
these “radials” are obtained every sweep, thus providing a 
complete picture of the weather in front of the aircraft. A 
radial can be defined as weather radar information along a 
line, out to a range, for a given tilt angle and scan angle from 
the aircraft as is known in the art. Image processor 135 
receives from the data stream radar image data correspond- 
ing to a weather radar image to be displayed on radar 
indicator or display device 140. The radar image data can be 
in any of a wide variety of formats, such as bitmaps for 
example 

In a conventional weather radar system, the data stream 
containing the radar image data corresponding to a weather 
radar image to be displayed is provided directly to radar 
indicator 140 for use in displaying the weather radar image 
to the pilot of the aircraft or operator of the vehicle. 
However, there are features of weather radar images that 
indicate hazardous conditions may exist, but which occur 
relatively infrequently. Because these features occur 
infrequently, pilots may not be able to maintain proficiency 
in correctly interpreting these features. Further, some of 
these features are short-lived, and pilots may not notice the 
potential hazard on the radar indicator in time. Because of 
the short-lived nature of some features, and the infrequency 
of others, the pilot’s situational awareness is less than 
optimal, sometimes resulting in a failure to take appropriate 
action. 

In order to improve the situational awareness of the pilot, 
in the present invention the weather radar data stream 
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corresponding to a weather radar image to be displayed on 
radar indicator 140 is provided to image processor 135 for 
hazard identification. Image processor 135 implements 
image processor algorithms to identify features which may 
indicate hazardous conditions. These features can include, 
for example, squall lines, hooks, steep gradients, scalloped 
edges, pendant shapes, fingers, horse-shoe shapes, and other 
types. Once identified, the weather radar image is displayed 
by radar indicator 140, and the pilot or other user is notified 
of the existence of the feature which is potentially indicative 
of the hazardous weather condition. 

The image processing algorithms implemented by pro- 
cessor 135 for hazard identification can utilize different 
image processing procedures and techniques of the type 
which are known in the art. For example, the image pro- 
cessing algorithms can use image processing procedures for 
two dimensional Fourier transforms, median filtering, edge 
detection, morphology, neural network implementation, 
and/or thresholding. In image processing, procedures such 
as these are used for feature extraction. The extracted 
features are then used in a classification or pattern recogni- 
tion stage. Procedures for doing pattern recognition include: 
maximum likelihood, discriminant functions, and neural 
network implementation procedures. Further discussions of 
known image processing techniques and procedures can be 
found, for example, in the following texts: (1) Two- 
dimensional Signal and Image Processing, by Jae S. Tim 
(Prentice Hall, ISBN: 139353224); (2) Mathematical Mor- 
phology in Image Processing, by Edward Dougherty (editor) 
and Marcel Dekker (ISBN: 0824787242); (3) Neural Net- 
works for Vision and Image Processing, by Gail A. Carpen- 
ter and Stephen Grossberg (MIT Press; ISBN: 0262531089); 
and (4) Pattern Classification, by Richard O. Duda, Peter E. 
Hart and David G. Stork (John Wiley & Sons, ISBN: 
471056693). The present invention is not limited to any 
particular image processing technique chosen for identifying 
hazardous features in a weather radar image. 

After image processing to identify a feature of the weather 
radar image which is potentially indicative of a hazardous 
weather condition, a wide variety of methods can be used to 
notify the user of the existence of the feature. In numerous 
embodiments described below with reference to FIGS. 2-8, 
the pilot or user is notified by displaying the weather radar 
image on radar indicator 140 with added emphasis of the 
feature. In some embodiments, this includes adding to the 
weather radar image a text warning of the existence of the 
feature. This can include, for example, displaying a name of 
the type of the feature. For example, upon the identification 
of a squall line feature 200 as shown in FIG. 2, the text 
warning 210 which reads “WARNING SQUALL LINE” can 
be displayed. The text warning of the existence of the feature 
can also include proximity information indicative of a 
position of the hazardous weather condition. For example, 
referring for the moment to FIG. 7, a text warning 710 which 
reads “WARNING HORSE-SHOE SHAPED HAZARD 
RANGE: 120 MILES” can be displayed. Further, as is also 
illustrated in FIG. 7, the text warning can also include 
potential implications of the hazardous feature detected. For 
example, the text warning can also include more specific 
information indicating the type of weather (e.g., convection, 
hail, drafts, etc) commonly seen around the particular storm 
feature. In FIG. 7, the text warning for the horse-shoe shaped 
hazard includes the weather implications specific warning 
“TURBULENCE, THUNDERSTORMS AND HAIL”. 

In some embodiments, either instead of or in conjunction 
with the display of a text warning, the radar image is 
displayed to the user with added visual emphasis of the 
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hazardous condition feature by displaying the weather radar 
image with a visual border formed around the feature. The 
visual border can be a solid border, a non-continuous border, 
a flashing border, or other types of borders to highlight the 
5 location of the hazardous feature. Any text warning com- 
bined with the border can be positioned inside of the border, 
or outside of the border. 

In addition to text warnings and visual borders, other 
embodiments of the present invention include other methods 
10 of adding emphasis to the feature in the weather radar image, 
as compared to a standard weather radar image in which the 
feature is illustrated but not emphasized. For example, the 
existence of a hazard indicating feature could be highlighted 
by causing an icon to vary in a visually distinctive manner. 
15 Alternatively, the feature itself can be caused to vary in a 
manner which is not typical in a conventional weather radar 
image. For example, the feature itself could be caused to 
flash by varying the intensity repeatedly. 

In yet other embodiments, notifying the user of the 
20 existence of the feature of the weather radar image which is 
potentially indicative of the hazardous weather condition is 
achieved using an optional warning device 145. Warning 
device 145 can be a non-visual warning device in some 
embodiments of the invention. For example, warning device 
25 145 can include a speaker for generating an audible tone 
once image processing circuitry 135 detects the hazardous 
feature in the weather radar image. Other non-visual warn- 
ing devices could also be used. Further, warning device 145 
can include visual warning devices which are separate from 
30 radar indicator 140. For example, warning device 140 could 
include a flashing warning light. 

FIGS. 2-8 illustrate cockpit display device or radar indi- 
cator 140 displaying hazardous condition enhanced weather 
35 radar imagery in accordance with various embodiments of 
the present invention. These FIGS, illustrate various haz- 
ardous weather condition features which can be detected, as 
well as a variety of formats for notifying or warning the pilot 
or user of the existence of the hazardous features. It must be 
4Q noted that the present invention is not limited to the par- 
ticular hazardous features illustrated, nor to the specific 
warning formats used with particular hazardous features. 
For example, the warning format used for one particular 
hazardous feature can be applied to another hazardous 
45 feature, or the warning formats can be combined and revised 
in numerous different fashions. 

Referring again to FIG. 2, radar indicator 140 is shown 
illustrating feature 200 indicative of a squall line. A con- 
tinuous line of storm cells, such as this, is generally 
50 impenetrable, and a detour of an aircraft around this type of 
area is recommended. Using image processing circuitry 135 
to detect hazardous squall line feature 200, text warning 210 
is generated and displayed. Text warning 210 can be in 
whatever color is determined to be most visually distin- 
55 guishable. Further, text warning 210 can be continuous or 
flashing. In the illustrated example, text warning 210 is 
surrounded by a border, but this need not be the case. 

FIG. 3 illustrates radar indicator 140 displaying a weather 
radar image including a hook feature 300. Hook feature 300, 
60 at approximately 30° right of center and approximately 7 
miles, out suggests the presence of hail and turbulence. In 
the embodiment illustrated in FIG. 3, hook feature 300 is 
emphasized using both a text warning 310 and a visual 
border 320. Once again, the text and border can be in any of 
65 a wide variety of formats. 

FIG. 4 is an illustration of radar indicator 140 displaying 
a feature 400 indicative of a steep gradient hazardous 
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weather condition. This storm cell indicates a large area of 
heavy precipitation surrounded by small areas of moderate 
precipitation. This type of activity commonly includes 
severe turbulence. Further, the lack of returns displayed 
behind this feature may indicate possible attenuation of the 5 
radar pulses. 

Using image processing in accordance with the invention, 
feature 400 indicative of the potential presence of a hazard- 
ous weather condition is visually emphasized using both a 
text warning 410 and a visual border 420. In this particular io 
embodiment, the text warning is located within border 420, 
instead of outside of the border. Further, in this particular 
embodiment, the text warning is shown in two separate 
sections at the top and bottom of the bordered area. 

FIG. 5 illustrates radar indicator 140 displaying a weather 15 
radar image including a feature 500 indicative of a condition 
sometimes referred to as a scalloped edge. This type of 
feature is indicative of heavy turbulence and possibly hail. 

In the embodiment illustrated in FIG. 5, radar indicator 140 
provides a mode of visual emphasis of feature 500 which 20 
includes only a visual border 520, but not a text warning. 
Also, for purposes of illustrating the wide variety of different 
visual notification formats that can be used, border 520 is a 
non-continuous border. 

25 

FIG. 6 illustrates radar indicator 140 displaying a feature 
600 indicative of a hazardous condition known as a “finger”. 
The particular shape of the feature 600 has been associated 
with thunderstorms and hail. In the particular embodiment 
illustrated in FIG. 6, after identification by image processor 
135, the weather radar image is altered to display text J 
warning 610. In this particular embodiment, text warning 
610 indicates “POTENTIAL HAZARD”, but does not 
specify the name of the hazard. Further, in this non-limiting 
example, no borders are used. ^ 

FIG. 7 illustrates radar indicator 140 displaying a hazard- 
ous condition feature 700 which is indicative of a weather 
phenomena frequently referred to as a horse-shoe shaped 
hazard. 

This type of hazard suggests severe turbulence, thunder- 40 
storms and hail. In the embodiment illustrated in FIG. 7, 
feature 700 is emphasized using both a text warning 710 and 
a visual border 720. In this particular embodiment, text 
warning 710, which reads “WARNING HORSE-SHOE 
SHAPED HAZARD RANGE: 120 MILES 45 

(TURBULENCE, THUNDERSTORMS AND HAIL)”, pro- 
vides an indication of the proximity or position of the 
hazardous weather condition as well as the associated 
weather implications. The text warning could also provide 
an indication of the direction of the hazard relative to the 50 
aircraft. 

Although the present invention has been described with 
reference to illustrative embodiments, workers skilled in the 
art will recognize that changes may be made in form and 
detail without departing from the spirit and scope of the 55 
invention. 

What is claimed is: 

1. A method of providing weather radar images to a user, 
the method comprising: 

obtaining radar image data corresponding to a weather 60 
radar image to be displayed to the user, wherein such 
obtaining includes 

transmitting from a receiver/transmitter a different 
radar pulse in each of a plurality of different 
directions, 65 

receiving radar pulses reflected off of weather 
phenomena, and 
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generating the radar image data from the received radar 
pulses; 

image processing the radar image data to identify a 
feature of the weather radar image which is potentially 
indicative of a hazardous weather condition; 

displaying the weather radar image to the user, including 
displaying on the weather radar image a text warning, 
adjacent the display of the feature, apprising the user of 
the existence of the feature; and 

notifying the user of the existence of the feature of the 
weather radar image which is potentially indicative of 
the hazardous weather condition, such notifying includ- 
ing displaying the weather radar image to the user with 
added emphasis of the feature of the weather radar 
image. 

2. The method of claim 1, wherein displaying on the 
weather radar image the text warning of the existence of the 
feature further comprises displaying proximity information 
indicative of a position of the hazardous weather condition. 

3. The method of claim 2, wherein displaying the prox- 
imity information further comprises displaying range infor- 
mation indicative of a distance between the receiver/ 
transmitter and the hazardous weather condition. 

4. The method of claim 3, wherein displaying the prox- 
imity information further comprises displaying an indication 
of actual weather implied by the feature. 

5. The method of claim 1, wherein displaying the text 
warning of the existence of the feature further comprises 
displaying on the weather radar image a name of a type of 
the feature. 

6. The method of claim 1, wherein displaying the radar 
image to the user with added emphasis of the feature further 
comprises displaying the weather radar image with a visual 
border formed around the feature. 

7. The method of claim 1, wherein notifying the user of 
the existence of the feature of the weather radar image which 
is potentially indicative of the hazardous weather condition 
further comprises generating an audible warning of the 
existence of the feature. 

8. A weather radar system comprising: 

receiver/transmitter circuitry configured to transmit a dif- 
ferent radar pulse in each of a plurality of different 
directions, to receive radar pulses reflected off of 
weather phenomena, and to generate radar image data 
from the received radar pulses; 

image processing circuitry configured to receive the radar 
image data and to identify a feature of a corresponding 
weather radar image which is potentially indicative of 
a hazardous weather condition, the image processing 
circuitry providing as an output image processed radar 
image data, the image processing circuitry configured 
to provide the image processed radar image data such 
that the display device displays the weather image with 
added emphasis of the feature of the weather radar 
image, the image processing circuitry further config- 
ured to provide the image processed radar image such 
that the display device displays the weather radar image 
data so that it includes, adjacent the display of the 
feature, a text warning of the existence of the feature; 
and 

a display device coupled to the image processing circuitry 
and configured to display the weather radar image 
combined with a notification of the existence of the 
feature which is potentially indicative of the hazardous 
weather condition. 

9. The weather radar system of claim 8, wherein the image 
processing circuitry is configured to provide the image 
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processed radar image data such that the display device 
displays with the weather radar image textual proximity 
information indicative of a position of the hazardous 
weather condition. 

10. The weather radar system of claim 9, wherein the 
image processing circuitry is configured to provide the 
image processed radar image data such that the display 
device displays with the weather radar image textual range 
information indicative of a distance between the weather 
radar system and the hazardous weather condition. 

11. The weather radar system of claim 8, wherein the 
image processing circuitry is configured to provide the 
image processed radar image data such that the display 
device displays the weather radar image combined with a 
name of the hazardous weather condition. 

12. The weather radar system of claim 8, wherein the 
image processing circuitry is configured to provide the 
image processed radar image data such that the display 
device displays the weather radar image with added empha- 
sis of the feature of the weather radar image by displaying 
a visual border around the feature. 

13. The weather radar system of claim 8, and further 
comprising a non-visual warning device coupled to the 
image processing circuitry and configured to generate a 
non- visual warning of the existence of the feature. 

14. The weather radar system of claim 13, wherein the 
non-visual warning device includes a speaker which gener- 
ates an audible warning of the existence of the feature. 

15. A weather radar system comprising: 

receiver/transmitter circuitry configured to transmit a dif- 
ferent radar pulse in each of a plurality of different 
directions, to receive radar pulses reflected off of 
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weather phenomena, and to generate radar image data 
from the received radar pulses; 

image processing circuitry configured to receive the radar 
image data and to identify a feature of a corresponding 
5 weather radar image which is potentially indicative of 
a hazardous weather condition, the image processing 
circuitry providing as an output image processed radar 
image data, the image processing circuitry configured 
to provide the image processed radar image data such 
10 that the display device displays the weather radar image 
with added emphasis of the feature of the weather radar 
image, the image processing circuitry further config- 
ured to provide the image processed radar image data 
such that the display device displays the weather radar 
image with added emphasis of the feature of the 
15 weather radar image by displaying a visual border 
around the feature; and 

a display device coupled to the image processing circuitry 
and configured to display the weather radar image 
combined with a notification of the existence of the 
20 feature which is potentially indicative of the hazardous 
weather condition. 

16. The weather radar system of claim 15, and further 
comprising a non-visual warning device coupled to the 
image processing circuitry and configured to generate a 

25 non-visual warning of the existence of the feature. 

17. The weather radar system of claim 16, wherein the 
non-visual warning device includes a speaker which gener- 
ates an audible warning of the existence of the feature. 

18. The weather radar system of claim 15, wherein the 
30 visual border is discontinuous. 



